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1.  Introduction 

A  major  contributor  to  the  life  cycle  cost  of  information  systems  is  die  cost  of  supporting 
these  systems  once  they  are  fielded.  The  support  of  existing  information  systems  has  histori¬ 
cally  been  perceived  as  a  less  glamorous  task  than  the  development  of  new  informaticm  systems. 
However,  it  has  beoi  estimated  that  support  costs  now  consume  greater  than  70  percent  of  total 
life  cycle  cost  and  that  this  proportion  is  still  rising  [24]. 

Unfortunately,  while  support  cost  has  become  an  increasingly  acute  problem,  diere  is  not 
yet  an  effective  means  for  determining  support  cost  drivers  and  reducing  support  cost.  Not  (xily 
is  the  support  process  still  poorly  understood,  there  is  no  comprehensive  set  of  quality  measures 
diat  may  be  utilized  to  understand  and  control  the  process.  Determining  appr(q)riate  quality 
measures  is  no  simple  task,  givoi  the  increasing  size  and  complexity  of  information  systems, 
the  potentially  large  number  of  staff  supporting  these  systems,  the  intricacy  of  infotmadmi  sys¬ 
tems  support  planning,  and  the  diversity  of  information  systems  users.  The  information  systems 
support  organization  must  have  a  method  for  evaluating  their  capability  to  adequately  support 
their  cttilection  of  Information  systems.  In  addition,  measures  of  the  supportability  of  informa¬ 
tion  systems  and  the  availability  of  diese  systems  to  the  users  is  needed. 

The  objective  of  the  Software  Support  Qualitative  Assessment  Methodology  (contract  no. 
ECD-8904815)  is  the  development  of  a  ffamewoik  of  measures  for  information  systems  support 
for  utilization  by  the  U.  S.  Army  information  systems  suppon  orgaitizations.  It  is  likely  that  a 
framework  of  measures  for  information  systems  support  can  be  developed  based  on  previously 
proposed  life  cycle  measures.  The  key  to  developing  this  measurement  framework  is:  (1)  under¬ 
standing  the  availability  of  current  metres;  (2)  constructing  a  valid  model  of  the  information 
systems  support  process;  and  (3)  developing  the  measurement  framework  around  the  proposed 
model. 
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In  ttUs  paper,  we  review  some  of  the  most  important  metrics  diat  may  be  applied  to  the 
infonnatitm  systems  support  cycle,  dividing  the  metrics  iiuo  diree  broad  classes.  We  then  out¬ 
line  an  infoimatitHi  systems  support  model  and  describe  top-level  measures  based  on  this  model 
which  are  built  from  the  reviewed  metrics. 

2.  Classes  of  Measures  for  Information  Systems  Support 

A  metric  is  a  measurable  indication  of  some  quantitative  aspect  of  a  system  [10].  We  use 
metrics  to  establish  an  indicator  of  merit  for  accurately  evaluating  software  and  its  development 
and  support  Metrics  are  also  used  to  control  and  predia  die  quality  of  software  and  the  process 
of  developing  and  supporting  this  software.  Metrics  may  be  thought  of  as  falling  into  one  of 
three  classes.  One  class  of  metrics  quantifies  attributes  of  the  software  product  via  static 
analysis  of  the  software’s  source  code  and  accompanying  internal  and  external  documentation 
Examples  of  attributes  in  this  class  are  complexity  and  modularity.  Anodier  class  of  metrics 
quantifies  aspects  of  the  software  develoixnent  and  support  process.  Still  another  class  of 
metrics  assesses  the  behavior  or  fimctionality  of  tire  software  based  upon  its  execution  (e.g.,  a 
reliability  metric). 

These  diree  classes  of  metrics  can  provide  useful  information  to  die  various  people 
involved  in  the  information  systems  support  process.  The  software  technician  is  most  interested 
in  producing  high-quality  software  and  thus  may  find  the  static  measures  of  product  quality  use¬ 
ful.  The  information  systems  support  administrator  interested  in  improving  the  support  process 
may  find  the  support  process  measures  most  useful  The  user  of  die  information  systems  is 
most  interested  in  issues  such  as  die  usability,  reliability,  and  availability  of  the  information  sys¬ 
tems,  thus  suggesting  die  usefulness  of  behavioral  metrics. 

In  the  following  three  sections,  a  review  of  the  three  classes  of  metrics  for  information 
systems  siqiport  is  given.  This  review  serves  as  foundation  upon  which  we  can  subsequently 
build  a  framework  of  information  systems  support  measures. 

3.  Software  Product  Metrics 

Software  product  metrics  measure  characteristics  exhibited  by  a  static  software  product 
(source  code,  documentation).  The  metrics  provide  a  method  of  measuring  product  quality. 
Although  the  ptodua  is  a  reflection  of  the  process  that  created  it,  the  metrics  do  not  provide 
enough  infotmatitm  to  accurately  analyze  the  process.  Software  product  metrics  are  of  particu¬ 
lar  importance  to  those  most  closely  associated  with  the  development  and  support  of  the  pro¬ 
duct. 


-3- 


The  characteristics  of  the  software  product  which  affect  the  ability  to  support  the  product 
are  those  detennining  the  mairttainability  of  the  product  Maintainability  is  fonnally  defined  as 
follows: 

Maintainability  is  the  ability  of  an  item,  under  specified  conditions  of  use,  to  be  retained 
in,  or  restored  to,  within  a  given  period  of  time,  a  specified  state  in  which  it  can  perform 
its  requited  functions,  when  maintenance  is  performed  under  stated  conditions  and  while 
using  prescribed  procedures  and  resources  [9]. 

Modification  of  software  is  a  non-trivial  task.  It  involves  such  activities  as  program 
comprehension,  diagnosis,  repair  or  enhancement,  and  testing.  Many  software  produa  design 
considerations  affect  the  ease  of  software  modification.  These  attributes  are  defined  in  the  fol¬ 
lowing  table. 


Complexity 

A  characteristic  of  the  software  interface 
which  influences  the  resources  anoftier 
system  will  experrd  or  commit  while 
interfacing  with  the  software  [8]. 

Coiuirtency 

The  extent  to  which  uniform  design 
techniques  and  notation  are  used  [25]. 

Modularity 

Characteristics  which  provide 
well-structured,  highly  cohesive, 
minimally  coupled  software  [25]. 

Self-Descriptiveness 

Characteristics  which  provide  an 
explanation  of  the  imfdementation  of 
functions  [25]. 

Testability 

-  -  - 

The  extent  to  which  software  facilitates 
both  the  establishment  of  test  criteria 
and  the  evaluation  of  the  software  with 
respea  to  those  criteria  [1]. 

Table  1;  Software  Design  Factors  Influencing  Maintainability 


Other  aspects  of  the  software  produa  can  affect  its  maintainalality.  Examples  include  the 
implementation  language(s)  and  the  size  of  the  produa.  Another  important  factor  in  predicting 
the  ability  to  perform  corrective  maintenance  tasks  on  a  software  produa  is  the  age  of  the 
software,  or,  more  directly,  the  extent  to  which  the  software  has  been  previously  modified.  A 
recent  study  found  that  83%  of  software  faults  were  a  result  of  modifications  performed  on  an 
already  fielded  product  [7],  suggesting  the  cntcial  role  post-release  modification  plays  in  the 
support  of  software. 
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To  summarize,  the  set  of  factors  that  appear  to  most  likely  influence  software  maintahut' 
bility  is  as  follows: 

Con^>lexity  Size 

Consistency  Programming  Language 

Modularity  Age 

Self-Descriptiveness  Modifications 

Testabiiity 


Table  2  lists  possible  objective  metrics  that  may  be  used  to  measure  the  attributes  of 
software  product  maintainability.  Not  all  attributes  are  listed  in  the  table.  Namely,  m^cs  for 
consistency  and  self-descripdveness,  two  attributes  for  which  objective  measurements  are  diffi¬ 
cult  to  obtain,  are  not  presented.  In  these  two  cases  (and  perhaps  others  where  it  may  be  diffi¬ 
cult  to  gather  objective  measures),  one  would  need  to  develop  an  additional  set  of  subjective 
metrics  to  measure  the  attributes.  For  instance,  we  may  measure  the  self-descripdveness  attri¬ 
bute  by  nodng  the  existence  and  adequacy  of  certain  types  of  documentadon  and  the  adherence 
of  documentadon  to  exisdng  standards. 


Attribute  Metric 


Complexity 

Cyclomadc  Complexity  [V(0)]  [19] 
Effort  Metric  [14] 

Information  Flow  [17] 

Logical  Stability  [27] 

Nesting  Level  [8] 

Reachability  [23] 

Logical  Complexity  [12] 

Span  Between  Dau  References  [IS] 

Modularity 

Cohesion  Metric  [11] 

Average  Module  Size 

Testability 

Reachability  [23] 

Cohesion  Metric  [11] 

Size 

Lines  of  Code  (LOC) 

Machine  Lang.  Instructions  [22] 
Number  of  Tokens  [14] 

Source  Code  Storage  Space 

Object  Code  Storage  Space 

Program  Volume  (V)  [14] 

Module  Count 

Average  Module  Size 

Procedure  Count  [5] 

Language 

Language  Level  [14] 

Age 

Program  Age  [5] 

Number  of  Program  Releases  [5] 

Modifications 

Total  Number  of  Modifications 

Rate  of  Modificatif  ns 

Table  2.  Examples  of  Software  Produa  Maintainability  Metrics 


From  the  examples  presented  in  the  Table  2,  it  is  clear  that  there  is  a  wide  assortment  of  metrics 
from  which  to  choose  to  measure  software  maintainability. 
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4.  Life  Cycle  Process  Metrics 

Life  cycle  process  metrics  measure  attributes  of  the  process  emi^oyed  by  an  information 
systems  organization  to  develop  and  suppoit  software.  Since  the  software  engineering  process 
cannot  be  measured  directly,  we  measure  the  elements  directly  contributing  to  the  execution  of 
the  process.  In  other  words,  we  measure  the  process  environment.  The  setting  of  the  process 
environment  is  the  information  systems  suppoit  organization.  Thus,  life  cycle  process  metrics 
usually  measure  some  attribute  of  the  development  or  suf^it  organization.  In  section  S  we  dis¬ 
cuss  process  metrics  not  directly  measuring  organizational  attributes. 

The  ability  to  successfully  execute  the  software  engineering  process  depends  on  two  key 
factors:  the  aUlity  to  successfully  manage  the  process  and  the  ability  of  resources  to  facilitate 
the  successful  execution  of  the  process.  The  life  cycle  process  metrics  thus  fall  into  one  of  the 
two  broad  categories  of  process  management  metrics  and  resource  metrics. 

4.1.  Process  Management  Metrics 

Process  management  metrics  are  the  least  well-defined  of  all  software  life  cycle  metrics. 
There  are  few,  if  any,  concrete,  objective  management  measures.  As  a  result,  most  management 
measures  are  subjective,  and  their  value  or  contribution  to  a  high-level  measure  is  difficult  to 
accurately  assess.  A  list  of  process  management  elemotts  [21]  are  as  follows: 


•  Project  Management 

Orgairizational  Structure 
Life  Cycle  Planning 
Design  Methods 
Implementation  Methods 
Testing  Strategy 
Project  Interfaces 

•  Configuration  Mangement 

Identification 

Control 

Audit/Review 


A  complete  evaluation  of  the  eftecdveness  a  process  management  element  involves  the 
measurement  of  three  characteristics  of  the  element  These  characteristics  are  existence*  usage* 
and  adequacy.  The  existence  characteristic  describes  the  presence  or  absence  of  an  element  of 
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process  management  believed  to  play  a  key  contribution  to  the  execution  of  the  software  pro¬ 
cess.  The  usage  characteristic  describes  the  degree  of  utilization  of  an  existing  process  manage¬ 
ment  element.  The  adequacy  characteristic  describes  the  effectiveness  of  the  particular  process 
management  element  that  exists  and  is  used. 

Metrics  for  ;.e:isuring  process  management  characteristics  are  not  as  easily  derivable  as 
product  metrics ,  4nce  there  is  no  tangible  product  to  measure.  The  exist^ce  characteristic  can 
be  expressed  u:.  a  binary  measure,  indicating  whether  a  certain  element  exists  or  does  not  exist 
The  usage  characteristic  may  be  expressed  in  terms  of  a  biirary  measure  (used  or  tmr  used),  or  it 
may  be  expressed  in  terms  of  die  degree  of  an  element’s  utilization,  if  such  information  is 
obtainable  (via  use  of  a  rank  scale  or  a  utilization  percentage).  Adequacy  is  difficult  to  measure 
because  it  is  completely  subjective.  One  method  of  consistently  measuring  adequacy  of  an  ele¬ 
ment  is  the  employment  of  a  rank  scale  (for  instance,  an  integer  scale  of  0  to  5,  where  0  s  com¬ 
pletely  inadequate  and  5  =  completely  adequate).  An  example  of  the  use  of  such  a  scale  is 
found  in  [20]. 

4,2.  Resource  Metrics 

Resource  metrics  measure  the  capability  of  resources  in  helping  to  fulfill  a  support 
organization’s  ability  to  support  its  systems.  The  collection  of  resources  is  represented  in  the 
two-dimensiorud  matrix  shown  in  table  3.  The  top  row  of  tenns  in  this  table  denotes  the  vari¬ 
ous  classes  of  resources,  while  the  left  hand  column  denotes  characteristics  we  are  interested  in 
measuring.  Thus,  we  have  nine  measures  of  interest:  personnel  availaUlity,  personnel  utiliza¬ 
tion,  and  so  forth. 


Persormei 

Software 

Hardware 

Availability 

Utilization 

Capability 

Table  3:  Resource  Matrix 


In  constructing  a  complete  set  of  resource  measurements,  if  one  wishes  to  utilize  objective 
metrics  to  the  maximal  extent  possible,  then  the  best  one  may  hope  for  is  a  mix  of  objective  and 
subjective  measures.  Table  4  shows  that,  for  some  resource  classes  (in  particular,  anything 
involving  personnel),  many  objective  metrics  have  been  pressed.  On  the  other  hand,  metrics 
for  many  of  the  classes  involving  hardware  or  software  are  much  harder  to  construct  Such 
metrics  are  dependent  upon  requirements  and  are  likely  to  be  at  least  in  part  subjective.  In  such 
cases,  we  once  again  would  want  to  use  the  existence,  usage  and  adequacy  characteristics  to 
obtain  complete  measures. 
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Characteristic 

Metric 

Hardware  Utilization 

CPU  Utilization  [22] 

I/O  Charmel  Utilization  [22] 

Memory  Utilization  [22] 

Test  Resource  Allocation  [16] 

Hardware  Capability 

Computer  Storage  Capacity 

Personnel  Availability 

Number  of  Project  Personnel  [3] 

Number  of  Support  Organization  Personnel 

Persoimel  Utilization 

Staff-Hours  [3] 

Percentage  Staff  Allocation  for  Task  [22] 

Personnel  Capability 

LOC/Production  Period  [22] 

Function  Points  [4] 

Education  Level  [3] 

Soft.  Eng.  Experience  [3] 

Soft.  Tech.  Experience  [3] 

Expert  Availability  [3] 

Organization  Experience  [3] 

Training  [3] 

Project  Experience  [22] 

Prxxluctivity  Quotient  [26] 

Programmer  Efficiency  [26] 

Software  Utilization 

_ 

Amount  (time)  Software  Usage 

Frequency  of  Usage 

Table  4:  Examples  of  Resource  Metrics 

5.  Behavioral  Metrics 

There  is  another  class  of  metrics  measuring  attributes  of  the  software  product  diat  may  be 
analyzed  only  after  the  software  has  been  dynamically  executed.  An  example  of  such  an  attri¬ 
bute  is  software  reliability.  It  is  Im^propriate  to  classify  such  metrics  with  the  other  life  cycle 
process  metrics  because  they  are  at  best  indirect  measures  of  the  software  development  and  sup¬ 
port  process.  Essentially  these  metrics  measure  the  overall  ^ectiveness  of  the  software 
engineering  process  without  actually  measuring  the  process  itself  [13].  Table  5  lists  the  attri¬ 
butes  which  behavioreU  metrics  measure.  These  attributes  are  adapted  from  [9,2S],  and  [13]. 


I 


Usability 
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Reliability 


Performance 


Integrity 


Availability 


The  effort  required  to  use  software  relative 
u>  the  effort  required  to  implement  the 
software  [25]. 

IThe  probability  that  software  will  not  cause 
the  failure  of  a  system  for  a  specified  time 
under  specified  conditions  [1]. 

A  measure  of  the  abUity  of  a  computer  system 
to  perform  its  functions  [1]. 

The  extent  to  which  software  will  perform 
without  unauthorized  access  to  code  or  data  [25]. 

The  probability  that  software  will  be  able 
to  perform  its  designated  system  function 
when  requited  for  use  [1]. 


Some  of  the  above  attributes  (reliability,  in  particular)  have  been  studied  in  detail,  and 
specific  metrics  have  been  proposed  for  these  attributes.  Others,  such  as  usability,  have  not 
been  as  closely  studied  and  are  not  as  well  specified.  The  difference  between  the  two  types  of 
attributes  appears  to  depend  upon  whether  they  are  primarily  objective  or  subjective  measures. 
At  one  end  of  the  spectrum,  there  are  many  objective  metrics  which  may  be  tqiplied  to  measure 
system  reliability  or  availability.  On  the  other  hand,  the  usability  of  a  system  depends  as  much 
on  human  factors  as  on  specified  requirements  (the  usability  metrics  shown  in  table  6  are  at  best 
indirect  measures  of  usability). 
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Attribute 

Metric 

Usability 

Amount  of  User  Training 

Frequency  of  Communication  with  Users 
Percentage  of  False  Maintenance  Requests 

Reliability 

Mean  Time  to  Failure  (MTTF)  [2]. 

Failure  Rate  [2]. 

Fault  Density  [2]. 

Defect  Density  [2]. 

Estimated  Number  of  Remaining  Faults  [2]. 

Testing  Sufficiency  [2]. 

Number  of  Maintenance  Requests  for  Corrections 
Rate  of  Maintenance  Requests  for  Corrections 

Performance 

Resource  Consumption  [13] 

Throughput [13] 

Response  Time  [13] 

Integrity 

System  Access  Control  [6] 

System  Access  Identification  [6] 

Availability 

Mean  Time  Betweoi  Failures  [9] 

Mean  Time  to  Repair  [9] 

Total  System  Operational  ("Up")  Time  [9] 

Total  Time  to  Complete  Maintenance  Actions  [9]. 

Table  6;  Examples  of  Behavioral  Process  Metrics 

6,  Top-Level  Measures  of  Information  Systems  Support 

From  the  examples  in  the  above  sections  it  is  apparent  that  there  is  a  myriad  of  software 
engineering  metrics  ftom  which  one  can  choose.  While  a  few  metrics  q)pear  to  have  become 
more  popular  than  the  other  metrics  (for  instance,  certain  metrics  to  measure  tiie  size  and  com¬ 
plexity  of  computer  programs),  there  is  no  agreement  upon  which  metrics  truly  provide  the  most 
accurate  and  impoitant  measures  [28].  Additioiudly,  the  contribution  of  Identifiable  attributes  to 
top-level  measures  such  as  software  maintainability  and  suppoitability  remains  unclear.  Empiri¬ 
cal  studies  have  thus  far  not  resolved  these  issues.  At  any  rate,  the  v^ue  of  a  particular  metric 
is  dependent  upon  the  environment  in  which  it  is  measured  and  the  person(s)  interpreting  the 
resultant  measure. 

In  our  particular  case,  we  would  like  to  determine  the  ai^ropriate  metrics  to  incorporate 
into  a  framework  for  the  assessment  of  information  systems  support  Before  we  can  do  so,  we 
must  develop  a  valid  information  systems  support  model  around  which  we  can  center  our  meas¬ 
ures. 


- 11  - 


6.1.  Information  Systems  Support  Model 

The  ability  to  support  a  software  system  involves  a  complex  combination  of  factors, 
including  product  attributes,  process  attributes,  and  reliability  attributes.  The  ability  to  develop 
and  implement  a  support  assessment  methodology  depends  on  the  ability  to  successfully  sort  out 
the  seemingly  chaotic  combination  of  support  faaors.  We  must  also  determine  the  perspectives 
for  which  we  should  base  a  high-level  support  measures. 

Lientz  and  Swanson  [18]  idendfled  three  distinct  populations  in  the  information  systems 
support  environment: 


•  Information  Systems  Support  Organization 

•  Information  Systems 

•  Information  Systems  Users 


A  single  high-level  support  measure  may  not  accommodate  all  perspectives.  The  support 
organization  is  most  lUrely  concerned  primarily  with  its  ability  to  support  its  portfolio  of 
software  systems,  suggesting  the  need  for  a  organizational  assessment  measure.  System  users 
are  more  concerned  with  the  availability  or  "operational  readiness"  of  a  software  system,  while 
the  technical  support  staff  maintaining  a  certain  software  product  are  particularly  concerned  with 
the  supportability  of  that  product  While  each  of  the  possible  measures  is  influenced  to  a  certain 
degree  by  various  software  product  and  proce^  attributes,  the  impact  of  particular  factors  on 
these  measures  will  vary. 

A  model  of  information  systems  support  is  shown  in  Figure  1.  This  model  is  an  adapta¬ 
tion  of  the  model  presented  in  [24].  The  model  depicts  the  duee  distinct  entities,  or  perspec¬ 
tives,  within  die  infotmadon  systems  support  environment.  Depending  upon  one’s  perspective, 
the  view  of  informadon  systems  support  and  its  problems  will  differ.  The  lines  connecting  the 
entities  in  the  model  illustrate  the  relationships  shared  by  die  entities. 

Two  observations  we  may  draw  from  the  model  to  aid  us  in  constructing  suf^rt  measures 
are: 

•  The  view  of  information  systems  support  varies  depending  upon  one’s  persrective. 
Thus,  certain  top-level  measures  may  be  more  valuable,  and  a  certain  set  of  low-Ievel 

»  metrics  may  contribute  significantly  to  the  measures. 

•  Because  there  are  relationships  between  the  information  systems,  su{^rt  organization, 
and  collection  of  users,  there  will  be  at  least  a  small  degree  of  commonality  among  aU 
top-level  measures,  even  If  such  measures  are  interpreted  differently. 
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In  our  model,  we  associate  a  top-level  measure  with  each  of  the  diiee  perqwctives.  The 
three  top-level  measures  are  supportability,  support  organization  assessmoit,  and  opera¬ 
tional  readiness.  We  define  and  discuss  the  modvations  for  the  use  of  such  tenns  in  the  fol¬ 
lowing  section. 


Figure  1;  Suj^xjrt  Model 


6.2.  Top-Level  Support  Measures 

We  have  proposed  three  top-level  information  systems  support  measures  to  describe  the 
information  systems  support  envirorunent  while  accorrunodating  each  of  the  support  perspec¬ 
tives.  From  the  information  system’s  point  of  view,  this  measure  is  supportability.  Supporta¬ 
bility  is  the  measure  of  the  adequacy  of  products,  resources,  and  procedures  to  facilitate: 

•  The  intended  operation  of  a  software  system  or  the  restoration  of  the  system  to  its 
intended  operatiorul  state. 

•  The  modification  of  the  software  system  to  meet  new  requirements. 
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For  each  information  system  sui^rted,  the  supportability  measure  is  intended  to  answer  the 
questions,  "Is  the  information  system  maintainable?",  and,  "Are  the  resources  and  procedures 
used  to  support  the  information  system  adequate?" 

From  the  su|^rt  organization’s  point  of  view,  the  measure  we  propose  is  die  sui^rt 
organization  assessment  A  support  organization  assessment  is  an  assessment,  conducted 
either  by  the  su{^rt  organization  or  an  outside  agent,  of  the  organization’s  ability  to  effectively 
support  its  information  systems  portfolio.  The  support  organization  assessment  answers  the 
question,  "Can  the  support  organization  adequately  keep  its  collection  of  information  systems  up 
and  running?"  Note  that  with  this  measure  we  treat  information  systems  collectively  rather  than 
individually. 

For  the  information  system  user’s  viewpoint,  we  propose  operational  readiness.  Opera¬ 
tional  readiness  is  the  ability  of  a  software  system  to  perform  its  intended  function  upon 
request,  based  on: 

•  The  current  operational  state  of  the  system. 

•  The  reliability  of  the  system. 

•  The  supportability  of  the  system. 

Operational  readiness  addresses  such  questions  as  "Is  the  system  up  and  running  when  I  need 
it?",  and,  "When  I  use  the  system,  can  I  expect  correct  results?"  With  operational  readiness,  we 
are  again  addressing  infonnation  systems  on  an  individual  basis.  In  additlonn,  supportability 
(one  of  our  other  proposed  measures)  contributes  to  operational  readiness,  but  user-oriented  pro¬ 
cess  measures  also  contribute  to  the  top-level  measure. 

In  previous  sections  we  reviewed  three  classes  of  metrics  for  information  systems  support: 
product  metrics,  software  engineering  process  metrics,  and  user-oriented  process  metrics.  For 
each  of  our  three  top-level  support  measures,  one  class  of  metrics  takes  on  primary  importance. 
For  the  supportability  measure,  the  maintainability  product  metrics  are  of  primary  importance, 
while  the  process  metrics  play  a  lesser  role.  The  support  organization  is  based  heavily  on  sup¬ 
port  organization  in'ocess  metrics.  The  operational  readiness  measure  is  derived  from  the 
usage-oriented  process  metrics,  although  the  other  sets  of  metrics  contribute  to  the  measure  as 
well. 


6.3.  Building  the  Top-Level  Measures 

The  approach  we  take  to  develop  the  top-level  support  measures  is  a  compromise  between 
the  incorporating  extensive  low-level  metrics  that  would  need  to  be  collected  to  provide  a  com¬ 
plete  albeit  complex  measure  and  the  specification  of  only  one  or  two  metrics  that  su[^sedly 
would  provide  sufficient  measurement  of  the  entire  support  environment.  On  the  orre  hand,  the 
collection  of  a  vast  amount  of  metrics  would  be  cumbersome  and  impractical  (and  probably 
unnecessary).  On  the  other  hand,  it  is  unlikely  that  a  single  metric,  such  as  a  program  complex¬ 
ity  metric,  wholly  predicts  the  sui^rtability  of  an  information  system. 
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We  specify  the  composition  of  top-level  measures  in  accompanying  documents.  The 
measures  are  developed  using  a  combination  of  subjective  and  objective  metrics,  the  objective 
metrics  being  partly  drawn  from  the  metrics  outlined  in  the  above  sections.  The  rationale  for 
selecting  a  paiticular  metric  to  comprise  a  measure  is  based  on  the  capability  to  gather  the 
metric  across  a  variety  of  information  systems  support  environments  and  the  predictive  value  of 
the  metric  in  those  environments. 

7.  Conclusion 

In  this  paper,  we  reviewed  existing  metrics  for  measuring  information  systems  support 
We  first  divided  the  metrics  into  three  classes.  These  classes  of  metrics  serve  as  a  basis  from 
which  we  may  develop  a  set  of  top-level  support  measures.  Three  top-level  measures  are  pro¬ 
posed  which  accommodate  the  various  perspectives  in  the  information  systems  support  environ¬ 
ment  modeled  in  Section  6.  We  may  construct  these  measures  by  using  a  combination  of  objec¬ 
tive  and  subjective  metrics  from  each  of  the  three  metrics  classes.  In  additim,  the  construction 
of  the  three  top-level  measures  require  differing  Interpretatians  of  the  role  each  class  of  metrics 
plays  in  the  construction  process.  The  combination  of  these  top-level  measures  provide  a  com¬ 
plete  view  of  the  support  envirexunent  and  separately  provide  valuable  information  to  the  vari¬ 
ous  support  audiences.  The  development  of  these  support  measures  is  a  necessary  step  in  the 
understanding  and  control  of  the  information  systems  su{qx>rt  process.  This  ability  to  under¬ 
stand  and  control  the  process,  in  turn,  leads  to  the  greater  ability  to  reduce  information  systems 
support  costs  and  provide  sustained  support  for  information  systems  in  the  face  of  constantly 
changing  user  requirements. 


( 
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